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Figure 1. The knowledge, skills, and professional behaviors outlined in this resource help provide educators with the tools they 
need to make data-informed decisions to improve student learning. 
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SLDS Data Use Standards 


Background 


The Need 


As state and local education agencies increasingly focus on serving educators with their data systems, a common challenge has 
arisen: identifying the critical knowledge, skills, and behaviors needed by teachers and administrators to use data effectively. 
Many states are creating data literacy and data use training programs for pre- and in-service educators without a common 
foundation on which to base the content. In addition, several state education agencies and educator preparation programs have 
begun communicating about how to create a stronger alignment between pre- and in-service training for educators regarding 
data use. 


Goal and Objectives 


Goal: To increase the effective use of data by teachers and administrators to support student learning and success. 
Objectives 


1. To provide a foundation for states’ development of data literacy and data use trainings. 
2. To inform and improve the articulation between pre- and in-service data training for teachers and administrators. 


Contents and Intended Audiences 


This resource details the essential knowledge, skills, and professional behaviors required by educators to effectively use data 
to inform instructional and programmatic decisions. Each of the three main sections is further divided into subsections that 
contain individual standards and associated clarifying information (see figure 2). Note that a few standards (e.g., Data Quality) 
appear in more than one section; these standards are indicated with a link icon (@D). 


Section —ir- Knowledge 


Familiarity with the nature of data and concepts underlying data use; includes 
the learning and theory that education communities need as a foundation for 
using data to improve educational outcomes 


See definitions for KEY TERMS starting on page 28. 


Subsection ———» K.| FUNDAMENTAL 


The basic information needed to know how to use the data 


Standard ——— K.1.A Question Formation 


Knows which questions can be answered with data and how to identify the nature and extent of the data needed to 
answer questions. 


Clarifying ————— > Question formation is the first step of data use, but it also needs to have the end in mind. For example, finish 

Information the statement “I want to know... .” From there, educators can build a better question that helps identify the data 
sources, ASSESSMENTS, VARIABLES, student subgroups, programs or interventions, and/or time periods needed to 
answet it. The key is that the question must cover what the educator wants to know. 


Figure 2. Data use standards are organized into sections and subsections. Each standard is accompanied by clarifying 
information to illustrate key concepts. 
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Each section concludes with one or two scenarios that illustrate the standards being applied by educators in real-world 
situations (see figure 3). A glossary of key terms is included to increase clarity. A companion publication, SLDS Data Use 
Standards: Standards in Practice, contains three case studies that illustrate standards across all three sections being applied by 
educators in real-world settings. 


KNOWLEDGE SCENARIO |: 
TRAINING SCHOOL LEADERS TO SET GOALS WITH DATA 


Note: Parentheses indicate the Knowledge standard that is illustrated. 


Scenario You are a professional development trainer who is leading a training session for a school’s leadership team to 
Information build their data knowledge around understanding and refining their school improvement goals. These goals focus on 
improved results on the school climate survey and increased graduation rates (Types of Measures). 
Rel t you inform leadership team members about the laws and rules that direct data use (Data 
elevan Privacy), discuss what is appropriate use of data for the day’s activities (Data Ethics), and have them sign a statement 
Standard agreeing to behave appropriately (Data Privacy). 
gtccing PProp y oy 


You focus the team members on the goals they have identified, explaining that the goals are good but that data can best 
be used to answer specific and measurable questions. You lead them in an activity where they break down the goals into 


Figure 3. Each section includes scenarios demonstrating how standards can be applied in real-world situations. 


This resource is intended to inform pre- and in-service educator training programs. The language is purposefully nontechnical 
to make it accessible for educators. 


Approach 


The content of this resource was developed by the Data Use Standards Workgroup of the Statewide Longitudinal Data Systems 
(SLDS) Grant Program. State agency staff from Hawaii and South Dakota generated the idea for the workgroup. An open 
invitation to participate was sent to all states via the SLDS listserv, and the workgroup was formed in November 2013. 


Timeline 
Beginning of Idea Recruit and Plan Release Phase | Release Phase II 
(August 2013) (November 2013) Resource (July 2014) Resource (July 2015) 


Discuss with SST Launch of Phase | Launch Phase II 
(Fall 2013) (January 2014) (February 2015) 


Figure 4. Timeline of the development of SLDS Data Use Standards resources. 


Phase I 


The workgroup launched Phase I in January 2014 with a two-day, in-person work session to define its goals, create subgroups 
to execute the work, and establish a project plan for creating this resource. Three subgroups were formed: Knowledge, Skills, 
and Professional Behaviors. Following the in-person work session, the full workgroup and each subgroup met monthly via 
webinar ot phone call to review progress made and discuss issues that needed to be resolved. 


4 SLDS Data Use Standards 


The workgroup conducted a review of national and state educator standards to identify those that include references to data 
literacy or data use. The content of this resource was then aligned to those existing standards. The workgroup also performed 
a literature review of research on educator data use to identify common barriers to and facilitators of effective data use. 
Appendix B contains the list of primary references. Phase I of the workgroup concluded in July 2014 with the release of 
version 1 of this resource. 


Phase II 


Phase I began in February 2015 with another in-person work session to define the scope, subgroups, and products that would 
be created. Prior to the launch of Phase I, workgroup members began vetting the Version 1 resource with more than 10 
external stakeholder groups composed of the resource’s key audiences. 


Phase II includes 


1. this revision of the Version 1 resource to: (1) reflect feedback from external vetting; and (2) expand the clarifying 
information for each standard across all three sections; 

2. one or two scenarios for each section of this resource that demonstrate the use of individual-level data and 
ageregate data in education settings; 

3. three case studies that illustrate all three sections of this resource based on work in member organizations; and 

4. criteria for and a call to the field to submit real-world case studies of how states have applied the knowledge, skills, 
and professional behaviors. 


A fourth subgroup, Case Studies, was created to lead items 3 and 4, above. In addition, an advisory council was formed of 
members who conduct research on educator data use. 


Authors 


This resource was created by the 27 members of the Data Use Standards Workgroup representing 14 states and Guam! and 
facilitated by the SLDS Grant Program State Support Team. Members represent state education agencies, local education 
agencies, regional service agencies, postsecondary institutions, and state P-20W+ (early childhood through workforce) 
longitudinal data systems. Appendix A contains the list of workgroup members. 


' California, Guam, Hawaii, Idaho, Illinois, Kansas, Montana, Nebraska, New Hampshire, New York, North Dakota, Oklahoma, South Dakota, 
Tennessee, and Washington. 


Knowledge, Skills, and Professional Behaviors tor Effective Data Use - Version 2, July 200 q 


ws Knowledge 


Familiarity with the nature of data and concepts underlying data use; includes 
the learning and theory that education communities need as a foundation for 
using data to improve educational outcomes 


See definitions for KEY TERMS starting on page 28. 


K.1 FUNDAMENTAL 


The basic information needed to know how to use the data 


K.1.A Question Formation 


Knows which questions can be answered with data and how to identify the nature and extent of the data needed to 
answer questions. 


Question formation is the first step of data use, but it also needs to have the end in mind. For example, finish 
the statement “I want to know... .” From there, educators can build a better question that helps identify the data 
soutces, ASSESSMENTS, VARIABLES, student subgroups, programs or interventions, and/or time periods needed to 
answer it. The key is that the question must cover what the educator wants to know. 


K.1.B Data Quality 


Knows that HIGH-QUALITY DATA are based on VALID data that are RELIABLE, accurate, TIMELY, and complete. 


To be valid and meaningful to educators, data must be many things, including accurate and reliable. For example, 
if an educator uses a rubric to grade an assignment, the rubric must accurately measure the skills being assessed 
(VALIDITY). If the same rubric is used by another educator, then both educators must interpret the rubric the 
same way so that students receive comparable grades regardless of who scored their assignment (RELIABILITY). 


SD See also B.1.A Data Quality 


K.1.C Types of Data 


Knows that data come in two main forms— QUANTITATIVE and QUALITATIVE—and that, within these forms, there are 
other categories. 


Qualitative data are not numeric and can include words, images, sounds, and artifacts. For example, through 
interviewing parents, it could be determined that the school climate was considered safe and the school supported 
career or college readiness. Quantitative data are numeric. For example, a review of students’ records could show 
that out of 152 students in the high school senior class, 107 are going on to a four-year postsecondary institution 
while 10 students are joining the military and 22 plan to attend a technical institution. 


K.1.D Types of Measures 


Knows various types and purposes of ASSESSMENTS and other MEASURES. 


5 SLDS Data Use Standards 


Five categories of data measures are commonly used by educators: achievement data, perception data, 
demographic data, school process data, and program data. These categories may be organized, grouped, or 
labeled differently depending on which data ANALYSIS approach is selected. These measures ate often used to 
understand a school’s impact on student outcomes. Educators then describe and interpret these measures using 
averages, frequencies, growth calculations, and many more methods. 


K.1.E Data Metric 


Knows that MEASURES can be broken down into data metrics, which are calculated for ANALYSIS and monitored for changes. 


Educators use vatious data metrics every day, from average daily attendance to grade point average (GPA). How 
these mettics are calculated is very important. For example, graduation rate is a common metric. Some education 
agencies calculate this rate by finding the percentage of seniors who graduated. Others look back to find out what 
percentage of entering freshmen graduated four years later. Understanding the calculations and differences in 
metrics is important to putting the data into context. 


K.1.F Data Sources 


Knows different types of data sources and the benefits and limitations of using each. 


Educators have many different types of data sources available to them. For example, educators may want to 
determine college-going rates for students. They can either work with their local institutions to gather data or 
seek out data from a third-party partner, such as the National Student Clearinghouse. Educators need to know the 
benefits and limitations of each data source so they can pick the source most appropriate for their purpose. 


K.1.G Data Representations 


Knows common ways of representing and reporting data through various types of tables, charts, graphs, etc. 


Sometimes educators want to observe data in a visual format. For example, graduation rate TREND data may be 
best displayed in a line graph, and achievement data may be best displayed in a stacked bar graph. However, when 
exact values ate needed, data tables may be the best option. 


SD See also S.5.B Representation and B.1.C. Representation 


K.2 PROCESSING 


The knowledge needed to understand actions that can be taken with data 


K.2.A Types of Analysis 


Knows basic types of ANALYSES and when to use them. 


A common error for educators is to confuse CORRELATION with causation. Just because two traits or 
characteristics are frequently found together does not mean that one of the traits causes the other trait to be 
present. For example, if data show that all sixth-grade boys at a given school who read Book A scored proficient 
on statewide reading ASSESSMENTS, it would be inappropriate to conclude that all sixth-grade boys would be 
proficient if required to read Book A. 
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K.2.B Data Analysis Tools 


Knows there are different technology tools that can help collect, store, and analyze data, and that each has advantages 
and disadvantages. 


As an educator’s data use skills develop, the types of data ANALYSIS tools available for use are likely to expand. 
While common software may be adequate for analyzing classroom datasets, users of large datasets benefit from 
specialized software designed to ensure the accuracy, flexibility, speed, and ease of analyses. It is important to 
consider which tool is most beneficial for the intended purpose before preparing the data for analysis. Using a 
rudimentary tool for complex analyses may introduce inadvertent errors, result in inaccurate FINDINGS, of incur 
additional work on the part of the individual. 


K.2.C Data Collection 


Knows that DATA COLLECTION can be performed using different methods and at different points in time. 


When deciding to collect data for ANALYSIS, understanding the foundational components—especially how the 
data will be used—is critical to determining how the data will be collected. Certain data types lend themselves to 
particular data collection methods. For example, surveys or interviews are more suitable than ASSESSMENTS and 
observations for collecting data about how safe students feel at school; however, assessments and observations 
are more suitable for collecting data about students’ academic performances. 


K.2.D Data Context 


Knows the circumstances and purposes for which data are collected. 


How data ate collected can change the meaning of the data. Rather than starting a new DATA COLLECTION, an 
educator might choose to use an existing dataset. Before using an existing dataset in an ANALYSIS, the educator 
should know why the data were collected and what limitations apply to those data. For example, using self- 
reported or anecdotal data collected at the beginning of the school year for pre-test or BASELINE purposes in place 
of culminating high-stakes proficiency scores for students would introduce conflicting purposes. 


K.2.E Data Format 


Knows that organizing data is important and affects how questions are answered. 


Different purposes for data ANALYSIS require data to be entered in different configurations accordingly. For 
example, if the question has a FORMATIVE purpose, the data need to be organized in a manner that provides 
actionable results (e.g., organized to produce individual student-level data versus classroom-level data). Educators 
benefit from knowing how the data need to be organized for the intended purpose and analysis tool prior to 
beginning their DATA COLLECTION. 


SD See also S.4.A Formatting 


K.3 CONSIDERATIONS 


The knowledge of best practices regarding data use 


K.3.A Data Assumptions 


Knows that assumptions affect the interpretation and usefulness of data. 
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There are many things an educator may take for granted when collecting or analyzing data. For example, an 
educator may assume people answer truthfully on a survey. In reality, some respondents may have inflated or 
deflated their responses based on how the survey will be used. Another educator may assume that a test accurately 
assesses students’ skills, but a student could have cheated on the test. 


K.3.B Data Limitations 


Knows that data have limitations and that these limitations affect the interpretation and usefulness of data. 


Even though data can help answer many questions, there are potential weaknesses in data that are not under the 
uset’s control. For example, when trying to measure student achievement, the data are only as good as the test 
itself. When conducting a parent survey on school climate, responses may be affected by the family’s ability to 
access the Internet as well as their language and reading skills. 


K.3.C Data Culture 


Knows best practices that contribute to a culture that values data and research-based solutions. 


Educators are encouraged by a culture that promotes effective data use within the classroom, across teams or 
departments, and throughout the school or district. For example, educators can set the example for others by 
incorporating data discussions into team or staff meetings. Educators can also stay informed of current research 
so they can implement ideas that worked for others into their own work. 


SD See also B.1.E Culture 


K.3.D Privacy and Security 


Knows federal laws, state laws, and district policies related to data PRIVACY, security, CONFIDENTIALITY, HUMAN SUBJECT rights, 
and appropriate use. 


There are many regulations and policies that govern how data are used and secured. One overarching federal 
law is the Family Educational Rights and Privacy Act (FERPA). This law protects the privacy of student education 
records and gives parents and students certain rights with respect to education records. Education researchers 
abide by ethical principles for the protection of human subject rights. This ensures that the rights, welfare, and 
wellbeing of students, teachers, and/or their data are protected if they are involved in academic research. 


K.3.E Data Ethics 


Knows ethical practices regarding the appropriate use of data when sharing information and reporting to others. 


While laws set the legal parameters that govern data use, ethics establish the fundamental principles of “right and 
wrong” that are critical to the appropriate management and use of education data. Local policies help enforce 
ethical practices, but ethics are also an individual’s professional responsibility. For example, an educator using data 
ethically would be careful not to make judgments about people based solely on data. He or she would also take 
steps to minimize any possible BIAS or errors in a data presentation. 


cD _ See also B.1.D. Ethics 
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KNOWLEDGE SCENARIO I: 
TRAINING SCHOOL LEADERS TO SET GOALS WITH DATA 


Note: Parentheses indicate the Knowledge standard that ts illustrated, 


You are a professional development trainer who is leading a training session for a school’s leadership team to 
build their data knowledge around understanding and refining their school improvement goals. These goals focus on 
improved results on the school climate survey and increased graduation rates (Types of Measures). 


At the beginning of the day, you inform leadership team members about the laws and rules that direct data use (Data 
Privacy), discuss what is appropriate use of data for the day’s activities (Data Ethics), and have them sign a statement 
agreeing to behave appropriately (Data Privacy). 


You focus the team members on the goals they have identified, explaining that the goals are good but that data can best 
be used to answer specific and measurable questions. You lead them in an activity where they break down the goals into 
specific questions (Question Formation). A goal related to high school graduation was refined into two questions. The first 
focused on the full student population: “For the past four years, what percentage of 12th graders graduated in four 
years?” The second focused on a targeted subgroup of students: “How does the graduation rate of English Language 
Learners compare to students who were non-English Language Learners for the past four years?” A goal to improve the 
school climate was refined to focus on differences between the general student population and those students who were 
chronically absent, or absent 10 percent or more of school days. The group agreed to the questions, “What are the trends 
in school climate for the entire school population over the past four years?” and “How does attendance correlate with 
school climate?” 


You ask the participants what data they have that tells them whether they met these goals (Data Sources). Leadership team 
members explain that they know which students graduate in four years because they already have a report in their student 
information system that is easy to understand (Data Forma?). The school climate survey is distributed to students in the 
spring of every year (Data Collection). Team members can access the results using multiple spreadsheets in Microsoft Excel 
(Data Analysis Tools). 


One person raised the concern that the school climate survey does not contain attendance data (Data Limitations). With 
these data unavailable, the group decided to drop the question that focused on the correlation between attendance and 
school climate. Another person expressed concerns that the graduation rate may not be accurate because of special 
education students who stay with the school until they age out (Data Assumptions). Acknowledging this, the group 
confirmed that the four-year graduation rate would still be the best, yet not ideal, measure as it is used by the district and 
state to determine supports and consequences (Data Context). Others expressed concerns about the accuracy of the school 
climate survey, as 30 percent of students did not take the survey (Data Quality). The group decided that it would discuss 
how to better increase the response rate next year. 


You explain that the next step is to determine what they need to do to analyze the data. Members agreed that the 
questions lend themselves to descriptive analysis, where averages and frequencies are used to help understand the data 
(Lypes of Analysis). One member suggested that the group could look at predicting the likelihood a student will graduate in 
four yeats based on whether they are an English Language Learner or not. Other group members expressed concern that 
such an analysis would exclude many other factors influencing whether or not a student graduates and could therefore be 
interpreted incorrectly. The group determined that this analysis would not be appropriate for the purposes of its work or 
for results reported to the school board (Data Ezhics). 
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KNOWLEDGE SCENARIO 2: 
TEACHING FUTURE EDUCATORS TO USE DATA IN THE CLASSROOM 


Note: Parentheses indicate the Knowledge standard that 1s illustrated. 


You are a professor who trains future teachers on how to use data. Your course is required as a part of the 
curriculum, so the course must apply to both general education teachers as well as content subject teachers, from arts to 
world languages. 


You come up with a classroom example through which you will lead the class over the course of the term to explain the 
process of data use from beginning to end. In the example, a high school World History teacher thinks certain students 
excel more than others. The teacher does not see any specific reason for this discrepancy in knowledge base and asks 

the question, “Is there a difference in the backgrounds of the students that seem to do better in this class?” (Question 
Formation). The teacher would then look at the school, the district, the state, and other sources of data to try to find valid 
and reliable data (Data Quality) about her students’ backgrounds. The teacher may find quantitative data that include 
grades in previous history courses or data on the number of history classes students have taken. The teacher could 
conduct a survey asking students how often they have seen similar information in the past. The teacher should also look 
at demographic characteristics like student gender and whether or not the student is receiving support services. Finally, the 
teacher could examine the types of history classes taken by the students in past years (Lypes of Data). 


Once you finish explaining the processes of question formation and data collection, you move through the example to the 
next stage of data use. For this class, you have chosen to use the data program SPSS to conduct the analysis (Data Analysis 
Tools). After preparing some fictional data based on the data elements mentioned above, you instruct your class to open 
their datasets and then instruct them on how to use graphs to visualize the data elements (Data Representations) and how to 
calculate averages to look at trends in the data (e.g,, “Do students who have taken two or more high school history classes 
before perform better, on average?”’). You reinforce that the teacher needs to be mindful of the different types of classes 
that were taken (Data Context) when looking at the results of this analysis (Lypes of Analysis). You further explain that there 
are many other types of analysis that can be conducted to incorporate all the data you have collected and to predict which 
students may have trouble in the class (Lypes of Analysis). You also explain that this sample of data could be used to make 


generalizations about student success in history outside of your classroom (L)pes of Analysis). 


Following the analysis section of the class, you conduct a section which covers how data can be misinterpreted because 
assessments may not measure actual knowledge (Data Assumptions), how data may not extend to some other students 
outside the World History teacher’s classroom (Data Limitations), and how the use of data should include an inclusive 
culture so that best practices can be followed by all data users (Data Culture). 


Finally, you close the class with a section on data privacy and ethics. You cover FERPA and other applicable state laws and 
also go through the process of when and how to apply to an institutional review board for human subjects protection 
applications. You emphasize that data use is a powerful tool in education and that the ethics surrounding data use need 
full respect. 
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Skills 


The ability fo access, collect, analyze, interpret, act on, and communicate 
about data using appropriate tools and representations in a manner 
appropriate for the educator’s professional role and responsibility 


See definitions for KEY TERMS starting on page 28. 


S.| PLANNING 


Strategizes for data collection and management 


S$.1.A Goals and Questions 


Identifies BASELINE MEASURE(S) atid poses questions that can be answered with data. 


Focus on setting goals that are specific (include who, what, when, where), measurable (numerically or 
DESCRIPTIVELY), achievable (considers resources needed and is realistic), relevant (consistent with the mission), 
and time-bound (has realistic deadlines). The baseline measure is the starting point, and the measurement is the 
means of determining whether the goal has been met. Once the goal is set, questions answerable with data can 


be asked to inform progress on the goal. 


$.1.B Alignment 


Aligns question(s), type of data needed, and measurement tools (e.g,, ASSESSMENTS, surveys, etc.) with goals and objectives. 


Once the question (matched with the goal) is asked, the data must be appropriate to answer the question and the 
measurement tool must be adequate to gather the data needed to answer the question. 


Objective | Question | Data | Measure 


All students will 

have at least one 
opportunity each 
month to practice 
expository writing. 


All students are 
proficient in 
expository writing. 


What is the Samples of Rubric to assess 
level of student expository writing the expository 
proficiency from all students. — writing. 

in expository 

writing? 


Table 1. Aligning data and measurement tools to a specific goal. 


$.1.C Data Management 


Develops and implements a consistent and ethical DATA COLLECTION plan with procedures for data management and 


DATA DOCUMENTATION. 


Educators can develop a data collection plan that includes who will collect data, when it will be collected, how 
it will be collected, how it will be stored and analyzed, what the data mean (data documentation), and what 
procedures will be used to ensure accurate data. Ethical considerations include such things as keeping individual 
student data confidential, putting mechanisms in place to ensure data security, and providing adequate training 
for those who collect and for those who analyze the data. 
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S.1.D Data Meaning 


Identifies different types of data and can explain specific DATA DEFINITIONS and how data are collected and formatted. 


Data definition means that educators can explain how the MEASURE was defined. For example, proficiency in 
reading might be defined as scoring above 75 percent on an identified reading test. Educators must understand 
how data were collected, what data were collected, how they were stored, and what the data mean. 


$.2 SELECTING 


Locates, accesses, develops, and evaluates data sources 


$.2.A Data Discovery and Data Acquisition 


Identifies and locates appropriate data sources and can access the data from various sources (e.g,, classroom, school, district, 
state sources) for DATA ACQUISITION. 


Data are available at different levels and in various formats. At the classroom level, teachers might access 
individual student data related to performance and demographics kept on a spreadsheet. The school might have 
data on factors such as school attendance and graduation data housed in a web-based platform. At the district or 
state level, there could be databases that include information from across schools. Different permissions might be 
needed to access information at different levels, and the educator knows who to ask about the existence of and 
how to get access to different types of data. 


$.2.B Critical Evaluation 


Knows how to perform CRITICAL EVALUATION on data sources for reputability, quality (including VALIDITY and 
RELIABILITY), relevancy, and ability to address the identified need. 


Educators look at sources of data with a critical eye and ask questions about the data and their source, including: 
Is the source of the data a reputable organization or institution? Are the data relevant to the information need? 
Does the instrument used to collect the data have adequate reliability? Is the instrument a VALID MEASURE of what 
it purports to measure, and was it used to collect data from the audience for which it was intended? 


$.2.C Development of Measures 


When necessary, designs ASSESSMENTS, tests, surveys, questionnaires, and other MEASURES in order to gather data appropriate 
to answer education questions. 


At times, educators may want to gather information using tools specific to their needs, and available measures do 
not provide the information of interest. For example, surveys for parents or questionnaires for students related 
to particular school factors, or assessments targeted to specific classroom instruction may need to be developed. 
When educators need these tools, they also require skills in such things as survey design to develop them. Any 
measurements developed should undergo CRITICAL EVALUATION for data quality purposes. 
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S.3 COLLECTING 
Uses appropriate technologies and methods in acquiring data 


S.3.A Facilitation 


Collects data in ways that ensure VALID, RELIABLE data and that minimize BIAS. 


Problems can occur while data are being collected. For example, if a teacher gives tests to different students or 
groups of students under different circumstances, such as providing hints to one group or having a study 
session immediately before the test for one group, the RELIABILITY of the data may be questionable. If an 
ASSESSMENT designed for use at the high school level is given to sixth graders, the VALIDITY of the assessment is 
in question. If questions on a sutvey use scales that are unbalanced (strongly agree, agree, slightly agree, 
disagree), the resulting data could be biased. 


$.3.B Technology 


Uses appropriate technologies to collect, access, and store data. 


Educators need to be able to use technology in various ways related to data. Examples include gathering data 
using online ASSESSMENTS, accessing district-level data via a web-based tool, or storing student information in a 
spreadsheet or database. 


$.3.C Multiple Measures 


Uses MULTIPLE MEASURES (c.¢., FORMATIVE, SUMMATIVE, GROWTH MEASURES, etc.), appropriately. 


Educators should not rely on a single MEASURE to determine student performance and must be able to connect 
information from various sources. For example, SUMMATIVE ASSESSMENT data should be compared to and 
interpreted in the context of FORMATIVE ASSESSMENTS and classtoom observation. Summative data should also 
be looked at in relationship to initial BASELINE data. 


$.3.D Modifications 


Makes appropriate modifications and ACCOMMODATIONS in MEASURES and collection conditions. 


Accommodations are used to enhance access for individuals, such as altering the testing environment or altering 
the ASSESSMENT format. Examples include providing extended testing time for a student with a learning disability 
ot allowing speech-to-text applications to be used by students with writing difficulties. Modifications change the 
expectations, such as shortening a test for some students or using a different scoring rubric or grading scale based 
on individual student needs. 


S.4 ANALYZING 
Exhibits the technical skills necessary to examine data 


S.4.A Formatting 


Formats data in ways that allow appropriate ANALYSIS. 


Educators can organize data in a pre-established format appropriate for the type of analysis to be conducted. 
Data can be in numeric format, such as in spreadsheets or databases, or in text format. It is important to translate 
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text data (agree/disagree scales for example, or yes/no) into numeric data to allow for QUANTITATIVE analysis. 
Labeling of data is important as well as indicating the type of data (date, currency, general number, percent, etc.) 
and if using general numbers, specifying the number of decimals to be used. Educators understand, for example, 
if wanting to compare pre- and post-test data (matched data), an identifier should match the data, or pre- and 
post-test data can be entered on the same horizontal row (for example in a spreadsheet). 


SD See also K.2.E Data Format 


S.4.B Data Cleaning 


Identifies data issues that might impact ANALYSIS (e.g., missing data, inaccurate data, etc.) 


Before beginning analysis, data must be examined to identify errors. For example, numerical data should be 
assessed for outliers by examining the range of values, where data falling outside the expected range could be 
data errors. For nominal data, DESCRIPTIVE analysis of all categories could help identify erroneous categories or 
misspellings of the same category type (e.g,, in an examination of gender you may find the following categories: 
female, femal, male, mail). Inaccurate data should be corrected prior to analysis. 


S.4.C Aligned Analysis 


Using appropriate technologies, conducts ANALYSES suitable for the type of data collected, the VARIABLES identified, and the 
questions or hypotheses posed. 


Identifying the percentage of students within performance categories on an ASSESSMENT of agetegate scores 
from a group is DESCRIPTIVE analysis and thus describes performance across a population. Descriptive analysis is 
the most common and simplest type of analysis. To identify relationships between MEASURES, a CORRELATIONAL 
analysis may be used. INFERENTIAL analysis will show significant group differences and PREDICTIVE analysis will 
demonstrate how one measure can be used to predict outcomes on another measure. Excel spreadsheets or 
statistical packages can be used for many of these analyses but will likely require training to use effectively. 


$.4.D Considerations 


Appropriately considers SCORE DISTRIBUTION, means, proportions, and other statistical and contextual information during ANALYSIS. 


In an analysis of group performance, a set of extremely high or extremely low scores can skew aggregate scores, 
such as a group mean, in a high or low direction, respectively, thus making the group average less representative 
of the group overall. This is a particular danger when scores are aggregated across smaller groups of individuals 
because each individual score will have more influence on the aggregate score (mean). In such cases, it may 

be more appropriate to identify the middle score (median) or the most common score (mode) of a group to 
understand the group’s overall performance. 


$.4.E Comparisons 


Compares various data for TRIANGULATION and progress monitoring (e.g, comparing BASELINE and current data, comparing 
numeric and verbal data, etc.). 


Data from multiple data sources should be examined for consistency in performance outcomes to help inform 
instructional and intervention decisions. Data can also be assessed over time to identify performance changes in 
relation to instructional and intervention programs put in place. QUALITATIVE (verbal) data can be compared to 
QUANTITATIVE (numeric) data to see whether the qualitative data support the quantitative FINDINGS. 
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S.5 INTERPRETING 


Constructs meaning from data within a particular context 


S.5.A Locating 


Locates data in tables, graphs, and charts. 


Educators need to be able to locate data presented in various forms. In the bar chart below (figure 5), the 
designators on the horizontal axis stand for specific subscales on a reading test for third grade. The red shading 
indicates students who did not meet standards on those subscales. Educators can identify which of the subscales 
had the highest percentage of students who did not meet the expectations. 
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Figure 5. Third-grade reading assessment scores, by subscale indicator. 


S.5.B Representation 


Recognizes when data are misrepresented or used in misleading or inappropriate ways. 


The educator is able to avoid representing data in misleading or inappropriate ways, such as using percentages 
with small sample sizes, or comparing school-level data using percentages between a large and a small school. 
Other examples of representing data in misleading ways would be using a graph where the y axis does not start at 
zeto, not providing a scale on the x axis of a graph, or removing outlier scores on a scatterplot. The educator is 
able to determine when others make inferences from data that were not intended for that particular purpose. 


GD See also K.1.6 Data Representations and B.1.C Representation 
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S.5.C Patterns 


Identifies patterns, TRENDS, and gaps in data and suggests reasons for their occurrence. 


Educators can look at data, see patterns and trends, and begin to think about potential reasons for those patterns 
that can then be explored further. For example, in the chart below, the green line shows the percentage of Limited 
English Proficient (LEP) students meeting or exceeding a test standard, while the blue line shows the percentage 
of non-LEP students meeting or exceeding the same standard. In looking at the pattern, it appears that data 

were not collected for LEP students prior to 2008. Between 2008 and 2011, there was improvement in the 
performance of LEP students so that it was similar to that of non-LEP students in 2011. 
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Figure 6. Proficiency rates of LEP and non-LEP students. 


$.5.D Congruency 


Interprets data CONGRUENT with statistical rules (e.g., SCORE DISTRIBUTION, impact of extreme scores, and relationship 
between sample size and GENERALIZABILITY). 


Educators interpret data such as mean scores by looking at more than just the scores themselves. For example, 
mean scores can be influenced by such things as extreme scores or bi-modal distributions (i.e., groups of students 
scoring at the high and low ends of a scale with few scores in the middle) and may not be an accurate reflection 
of average performance in those situations. Educators do not make statements generalizing data to a larger 
population without ensuring that there was an adequate sample size and that the sample was representative of the 
larger group. They do not imply a CAUSAL relationship based on CORRELATION data. 
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S.6 COMMUNICATING 


Conveys information about data 


S.6.A Presentation 


Presents and displays data in various forms, both visually and in text, using appropriate technologies. 


Educators must be able to prepare such things as frequency tables, bar charts, line graphs, pictographs, and other 
visualizations, as well as summaries of QUALITATIVE data and text-based descriptions of data using standard and 
accepted practices in the field. They must use appropriate technologies (e.g., Microsoft Excel or chart functions in 
word processing programs) to develop professional-looking representations of data. 


S.6.B Explanation 


Explains different data representations and distinguishing features (e.g., histograms, bar charts, contingency tables). 


Educators should be able to explain to others how different ways of representing data are used for different 
purposes. For example, line graphs can show change over time, pictographs can show frequencies or compare 
items or ideas, pie charts can show percentages, scatter plots can show relationship direction and strength of 
relationships, etc. 


S.6.C Multiple Audiences 


Communicates effectively about data, interprets FINDINGS, and explains progress toward goals to a variety of constituent 
groups (e.g., students, families, and colleagues). 


Educators must be able to explain data and interpret findings in different ways to meet the needs of different 
audiences. For example, how a school communicates with parents and the community about third-grade reading 
performance may be different from how teachers communicate with one another about the same data. More 
simplified representations of the data with brief descriptions might be provided for public consumption, while 
the teachers may be presented with more detailed information such as performance on subscales, performance 
ranges, and differences in performance across different groups. 


S.7 ACTING 
Employs appropriate strategies based on findings 


S.7.A Strategies 


Identifies appropriate strategies grounded in evidence to address the needs and goals identified during data ANALYSIS. 


Educators use high-quality sources of information to help identify potential strategies to address needs or to 
reach goals. These sources can include studies reported by reputable sources, government reports, or strategies 
supported by theory or logic models. 
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S.7.B Action Plan 


Develops and implements an action plan that includes provisions for evaluating the plan’s effectiveness, and can clearly 
articulate the link between the data, FINDINGS, and plan to appropriate audiences. 


Educators can explain to others the link between the data (for example, male students score significantly lower 
than female students on a third-grade reading ASSESSMENT), an interpretation of the data (literature suggests that 
male students are engaged by different types of reading material than female students), and a choice of action (the 
educator plans to use more non-fiction and graphic novels in reading instruction). Educators can also develop 

an action plan that explains who will do what in a specific timeframe, what resources are needed for each action, 
potential issues in implementation, and how progress will be evaluated, as shown in the sample plan below. 


Goal: To improve reading performance of third-grade male students 


AY Coli Resources 
7 Xos io) a) (=) °) Timeline =X oYo} abt] 0) (=) LN (X= [=Yo] Possible Issues 


Include in our June — July ¢ Curriculum ¢ Funds to Determining 
second- and third- Curriculum specialist specialist purchase which books will 
grade reading identifies appropriate ® Second-and materials be removed from 


the curriculum to 
make room for 


curriculum reading materials. 


third- © Professi | 
nonfiction books ird-grade rofessiona 


focused on sports August eee oo the new materials. 
and hobbies/skills Materials purchased. ‘eannae 


(e.g., carpentry, 
mechanics) and 
graphic novels. 


September - November 
Curriculum specialist 
works with teachers to 
develop lesson plans. 


January - June 
Teachers implement 
new lesson plans. 


Table 2. Sample action plan to improve reading scores for third-grade male students. 


Evaluation: Male and female students’ scores on in-class assignments will be compared. State test 
scores in reading will be reviewed to determine if the performance gap has decreased. 


SKILLS SCENARIO: 
IMPROVING READING SCORES FOR THIRD-GRADERS 


Note: Parentheses indicate the Skills standard that ts illustrated. 


You are a teacher at a small elementary school with a principal and 10 teachers. All of you work together to make 
decisions about how to improve the performance of students at your school. 


In fall 2014, you review summative data from the state’s required annual testing for the past few years and notice that 
third-grade scores on the state reading assessment from 2012, 2013, and 2014 are lower than desired, with only about 60 
to 65 percent of students meeting or exceeding the standards. As a group, you decide that by the end of 2016, at least 70 
percent of third-grade students will meet or exceed the state reading test standards (Goals and Questions). 


To better understand potential explanations for the problem, you and your colleagues come up with a set of questions 
answerable with data (Goals and Questions). Are there common demographic characteristics among students who did not 
perform well on the test? Are there certain parts of the test—or subscales—where students did less well? You take these 
questions and develop a table to align the questions with the data needed to answer them (A/zgnment) and discuss where 
the needed data can be found (Data Discovery and Data Acquisition). 
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Improve reading e Are there ¢ Student ¢ District has state * State reading 
performance of common performance test data and test scores for 
students in Grade 3 demographic data matched the ability to different groups 
so that by the end characteristics to demographic match it with (male/female; low 
of 2016, 70 percent among students data. demographic socioeconomic 
meet or exceed who did not information. status / not low 
standards perform well on on the reading «The school's socioeconomic 
the test? test. state test report status; Limited 


e Were there contains English Fioficient / 
certain parts of sub-score results, NON-Limited English 
the tests where Proficient) 
students had State reading test 
problems? subscale scores. 


e Subscale scores 


Table 3. Questions, data, and measures related to the goal of improving third-grade reading proficiency. 


The baseline measure is the 64 percent of students whose scores meet or exceed the standards in reading overall on the 
2014 state test. You plan to compare this baseline to results in 2015 and 2016 to determine progress toward the goal 
(Comparisons). You also want to compare subscale information. The group looks more closely at the subscale data and 
locates key information and patterns in both table and graph forms (Locating, Patterns) and can explain to one another the 
key features of each (Exp/anation). You can describe what each score actually measures (Data Meaning). For example, the 
subscale on reading comprehension is determined by how many of the 10 comprehension questions a student answers 


correctly after reading a passage. 
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Figure 7. Third-grade reading assessment scores, by subscale indicator. 
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You review a bar chart of student performance on reading assessments (see figure 7, previous page) and see that for 
subscale R.3.3.2.1 (reading comprehension), 68 percent of students did not meet the criteria. The pattern indicates that 
reading comprehension is an area where students are performing well below other areas, which answers your question 
about whether there are certain parts of the test where students did less well. You also know that the distribution of 
scores is a key consideration when interpreting data, so you review the students’ score distribution for the 10 items that 
make up the comprehension score. You notice that the majority of items show a skewed distribution, with most students 
scoring at the lower end. 


Percent Meeting or 
Area @} 0} (=Yoa (1-9 Number of Items eee KY (ol ale lol cok 


Reading Overall Recalled 


Vocabulary R.3.2.1.1 
Development 


Word Identification Roee2 2 | 
Reading Strategies R.3.3.1.1 
Reading Comprehension Resroe2al 
Reading Fluency R.33.3.1 
Literary Elements RAL Mel 
Literary Techniques R.3.4.2.1 
Literature R.3.4.3.1 


Table 4. Third-grade reading results from 2014. 


Based on this interpretation, you decide to develop an action plan for the teachers in grades 1 through 3 to work together 
to focus on improving reading comprehension (Action Plan). You consult the district reading specialist, search the Internet, 
and review some research articles to determine how you might address reading comprehension (Strafegies). Your research 


identifies several strategies that have proven effective for improving reading comprehension, including comprehension 
monitoring, cooperative learning, and the use of graphic organizers. 


Your action plan includes monitoring comprehension development using formative assessments developed by the teachers 
in each grade level. You decide to use a technology tool to collect these formative assessments across classrooms to ensure 
consistency and to review data together to minimize bias (Technology, Facilitation). You realize that two students who have 
writing difficulties may need accommodation allowing the use of speech-to-text software for the assessment (Modhfications). 
You also decide to implement cooperative learning strategies to improve comprehension. 


In addition to the formative assessments focused on comprehension, you and your colleagues agree to assess the students’ 
overall reading levels each year using a standardized test to look at their growth across the three grades. You select four 
possible tests to review and go through a process of critical evaluation before deciding on one you believe best meets 
your needs and has the most evidence of validity and reliability. You also agree that the participating teachers will develop 
two data presentations each year to share the findings (Presentation), one with more detailed analysis for the teachers and 
principal and one to share with parents and students (Mu/uple Audiences). 


Back in your classroom, you decide that in addition to reviewing each student’s formative assessment and growth data, 
you will look at data from the cooperative learning groups since you know that using multiple measures can help better 
understand where issues are occurring, You ask your third grade collaborative groups to use an online discussion tool that 
allows you to monitor their discussions. You develop a rubric as a measurement tool (Development of Measures) to teview 
the discussion and assess the depth of comprehension exhibited by each student. You will score the group discussions 
and individual contributions and maintain the scores in an Excel spreadsheet (Data Management, Technology). You know 
that as you develop your spreadsheet, you need to consider the best formatting for the data that will allow analysis later 
and you must consider the type of data you will be collecting (e.g, nominal, ordinal, interval, or ratio) to ensure that your 
analysis is aligned. 
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Professional Behaviors 


Habits of professional action based on values and beliefs that underlie an 
educator's practice as it is related to data use 


See definitions for KEY TERMS starting on page 28. 


B.| ETHICAL USE 
Commits to the proper use of data 


Educators collect and use data about students, staff, and schools. Laws set the legal parameters that govern 

data use, and ethics establish fundamental principles of right and wrong in the management and use of data. 
Especially in the age of technological tools, it is important that educators understand and practice ethical data use. 
Throughout this section, strong words may be used, such as “should” and “essential.” The intent is to emphasize 
that educators need to interact with data in a way that honors laws, protects CONFIDENTIALITY, preserves an 
educator’s integrity, and encourages appropriate use of data to improve instruction. 


B.1.A Data Quality 


Strives to use, contribute, and maintain HIGH-QUALITY DATA. 


Elements of data quality include consistency, accuracy, integrity, relevancy, timeliness, completeness, VALIDITY, 
and RELIABILITY, which can be affected by multiple factors in the ASSESSMENT, collection, and ANALYSIS 
processes. Data quality can be supported through well-developed assessments, assessments used for their 
intended purpose, representative samples, careful collection, and well-defined analysis procedures that involve 
clean and accurate data. For example, district officials may wonder if the number of tardies and absences 
correlate with student grades earned in classes. The superintendent could call for all schools to submit student- 
level data, including tardies and absences by course and course grades. However, each school might define 
tardies and absences differently, creating a data quality problem. 


SD See also 6.1.B Data Quality 


B.1.B Transparency 


Ensures the transparency of the collection, ANALYSIS, and sharing of data. 
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Educators clearly communicate to appropriate stakeholders the manner and extent to which data will be collected, 
analyzed, reported, and distributed. For example, after determining an appropriate intervention for a student, 

an RTI (response to intervention) team member who is the student’s primary teacher may send a notice to the 
student’s parents or guardian explaining how the intervention was determined and what ASSESSMENT data were 
used to make the determination. 
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B.1.C Representation 


Avoids misleading, ambiguous, or inaccurate representations of data. 


Educators who participate in training related to presenting and reading data become educated consumers of data 
and know how to share data accurately and professionally with others. For example, when presenting results as 
percentages, an educator should indicate the population or sample size. If state ASSESSMENT results show that 50 
percent of assessment items were answered correctly for a standard, it would be misleading not to provide the 
total number of items, especially if the number is low. 


GSD See also K.1.G Data Representations and $.5.B Representation 


B.1.D Ethics 


Promotes the ethical use of data among colleagues. 


Educators participate in training about their ethical responsibilities in accessing, using, sharing, and managing 
education data. A culture of PRIVACY and security is established with specific procedures and protocols that all 
members of a school community follow. For example, a school principal facilitates a staff meeting that includes 
administrators, teachers, paraprofessionals, and office staff. Collectively, the school staff identifies key principles 
of ethical conduct and develops specific policies and procedures for specific actions to take if the ethical code is 
violated. If a teacher inadvertently leaves student-level data on a printer that is accessible by students, a follow-up 
procedure for a first breach may be a notice from the principal reiterating proper security procedures. Hach 
member of the school community develops a job-specific code of ethics that applies to individual work situations. 


For more information, see The Forum Guide to Data Ethics from the National Forum on Education Statistics, 


http:/ / nces.ed.gov/ pubs2010/ 2010801. pdf. 


GSD See also K.3.E Data Ethics 


B.1.E Culture 


Ensures that data use is grounded in a framework of values and norms. 


Data use is complex work. A culture of data use is established through leadership, norm setting, and time devoted 
to data use. Individuals working in isolation within a school or organization may work for a short-term project but 
is not sustainable and will not lead to organizational improvements. 


GSD See also K.3.C Data Culture 


B.1.F Use 


Reports any known data misuse to the proper authority. 


Educators understand and follow locally determined protocols (e.g., those developed at the school, district, or 
state level) for documenting and communicating concerns about DATA COLLECTION, ANALYSIS, reporting, and 
distribution, as well as for destroying data properly. Professionals also identify appropriate authorities, as per the 
protocols, to contact regarding concerns of data misuse. For example, concerns regarding data use should be 
communicated through a chain of command (see figure 8, next page). Issues should be resolved efficiently at the 
lowest possible level and judiciously so that the rights and PRIVACY of all individuals are protected. 
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Figure 8. Communication tree for data use issues. 


B.2 RULES AND REGULATIONS 


Acts in accordance with the legal, social, and economic considerations involved in the use of data 


B.2.A Rules and Laws 


Acts according to national, state, and local organizational rules and laws regarding the collection and use of data 
(e.g, HIPAA, FERPA). 


Educators ate aware of the rules and laws governing data use and sharing, To the extent possible, there should 

be a petson in every district or school who has high-level and current knowledge of relevant rules and laws to 
guide ethical and legal use of data. For example, results of individual student ASSESSMENTS can be shared only 
with those who have an educational right to know, such as the student, educators at the school who work with the 
student, and the student’s legal guardian(s). 


B.2.B Protection 


Protects HUMAN SUBJECTS from HARM during and after using subjects’ data. 


Educators understand and follow federal and state laws and regulations in maintaining, using, and releasing 
student information. They protect the integrity and accuracy of student data and demonstrate CONFIDENTIALITY 
and PRIVACY at all times. For example, in addition to FERPA requirements, the Individuals with Disabilities 
Education Act (IDEA) provides privacy protection to students who are receiving special education services. 
Additionally, as schools use third-party applications to assist with classroom instruction, it is important that 

a district employee reviews the terms of service to ensure the third party meets any and all state and federal 
requirements. Every school must have one individual responsible for 


1. ensuring the confidentiality of any personally identifiable information (PII); 

2. training all personnel who are collecting or using PII on policies about confidentiality, privacy, and 
FERPA; and 

3. maintaining a current list of all personnel who have access to PII. (34 CFR 300.572) 


See the Forum Guide to Protecting the Privacy of Student Information: State and Local Education Agencies, 
http:/ / nces.ed.gov/ pubs2004 / 2004 330.padf 
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B.2.C Advocacy for Protections 


Advocates for changes to existing laws or policies when needed to ensure the proper protection and use of data. 


Occasionally, it might be necessary to update or change existing laws or policies in order to adequately protect 
students and educators while maintaining the ability to use their data for continuous improvement. Creating a case 
for improving existing laws to better protect students and educators can help policymakers understand the nature 
of and need for changes. 


B.3 COLLABORATION 


Facilitates a collective effort to Use and share data 


B.3.A Collaborative Use 


Works effectively with colleagues to collect, analyze, and disseminate data. 


Collaborating with others to collect, analyze, and disseminate data can provide additional perspectives, lessen 
redundancies, allow for shared workload, and create a systematic approach to continuous improvement. For 
example, a Professional Learning Community (PLC) team meets to analyze students’ performance on state 
ASSESSMENTS. To help focus collaboration, a protocol is established that outlines exactly which data to examine 
in the reports, what the data mean, and how to measure students’ performance levels. This sets the stage for 
identifying intervention strategies that support student learning, 


B.3.B Collaborative Climate 


Creates and supports a climate of trust and CRITICAL REFLECTION in order to engage colleagues in challenging but 
PSYCHOLOGICALLY SAFE conversations about data. 


Team members must be honest with one another. Being honest includes being forthcoming with information, 
forgoing exaggerations, demonstrating personal integrity, and being willing to acknowledge mistakes and address 
any “elephants in the room.” All members must be willing to share information and ideas, as well as to listen 
respectfully—through both verbal and nonverbal communication—and consider differing perspectives and 
solutions offered in discussions. Establishing group norms allows for consistent expectations as to how group 
members interact with one another. All of these elements establish trust—the hallmark of a collaborative climate. 


B.3.C Outreach 


Consistently and actively seeks professional, community, and technological resources within and outside the organization to 
support ANALYSIS, CRITICAL REFLECTION, and problem solving. 


Reaching out to others and seeking resources is a way for educators to introduce new ideas, test ideas of their 
own, and keep current with regional, national, and international ideas that can positively affect their practice and 
the practice of others. Outreach also helps to perpetuate a collaborative climate and continuous improvement 
within an organization. 


B.3.D Prioritization 


Prioritizes time to analyze and use data. 


Schools are constantly balancing multiple priorities. Data use can be integrated into existing school initiatives or 
incorporated into new initiatives. Leaders should encourage the use of data during staff or team meetings. All 
educators should view data use as a necessary part of teaching. 
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B.4 CONTINUOUS IMPROVEMENT 


Embraces the challenge of evidence-based, continuous improvement and change through the use 
of data 


B.4.A Problem Solving 


Views challenges related to data as opportunities rather than as problems. 


Many challenges exist on the way to creating systems in which educators effectively use accurate MEASURES. 
Creativity, collaboration, and innovation can address perceived barriers (e.g, access to accurate and TIMELY data, 
establishment of data teams, or fiscal and human resource allocation) and lead to the design of efficient processes 
and solution-oriented results. 


B.4.B Improving Outcomes 


Seeks to understand and use data to improve and advance outcomes for oneself and others over time. 


Data are collected and used to demonstrate improvements for both students and educators. For example, prior to 
implementing a new program, educators can discuss the data needed to measure the program’s impact on educator 
or student achievement. They can then create a plan to collect and use data to improve program outcomes. 


B.4.C Professional Development 


Recognizes needs and aligns professional development for oneself and others to build skills around data use. 


Schools that embed knowledge, skills, and professional behaviors with data use throughout the organization 
promote a continuous cycle of learning. Staff meetings can include time for knowledge building; district and 
school professional development plans can contain data use training; and educator preparation programs can 
contain data use instruction. Educators should build data use training into their own plans. 


PROFESSIONAL BEHAVIORS SCENARIO: 
PLANNING INTERVENTIONS FOR SEVENTH-GRADE MATH STUDENTS 


Note: Parentheses indicate the Professional Behaviors standard that is illustrated. 


You are a teacher at a small suburban middle school that has launched data teams and an intervention initiative 
to provide targeted math instruction for all seventh-grade students regardless of their performance level. Your school has 
scheduled intervention blocks four days a week for targeted instruction (Problem Solving). 


Leadership and ownership of data use practices must start with the building principal, and all educators involved in the 


process must collectively build a culture that creates a safe environment to use data (Cu/ture, Collaborative Climate). To 
encourage data use, the principal has prioritized staff and student time for the interventions, ensured that educators have 
scheduled planning time to analyze data (Prioritization, Collaborative Use), and required staff development activities related to 
action research or other aspects of data use (Professional Development). 


The school’s intervention team includes a cross section of educators beyond the seventh-grade math teachers (CoM/aborative 
Use), as many of the concepts in seventh-grade math can be reinforced in other subjects and additional educators are 
often required to assist with interventions (Problem Solving). Special education teachers, other core content teachers, 

and teachers of art, music, and physical education offer professional input on students they serve and help brainstorm 
interventions that might apply not only to math classes, but also to other classes. 
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Your first intervention team meeting is spent establishing and agreeing to meeting norms. Setting norms helps to guide 
and focus your team’s work and create a positive working environment (CoM/aborative Climate, Ethics, Culture). Your team 
then determines which assessment data will be used to identify individual student needs, spends time analyzing and 
interpreting the data (Data Quality), and establishes how the data should be used (Data Quality, Representation). 


When considering the use of assessment data, your team discusses the fact that national assessments often are not 
intended for skill analysis at the student level. It may be that a national assessment can measure overall grade-level 
competency (Aggregate Data) but is not valid or reliable at the skill level (Representation). You decide to reach out to 
assessment experts to understand the purpose and validity of the data from specific assessments (Professional Development). 
The school principal allows you to take the necessary time for this work and incorporates reports, updates, and 
conversations about this effort at staff meetings, parent meetings, and other meetings (Cw/ture, Outreach, Prioritization). 


One of the challenges of determining interventions is effectively using data to accurately identify the needs of each 
student (Problem Solving). You recognize that it is important for your team to have data that can evaluate the full spectrum 
of students, from those needing remediation to students needing more challenges. Once you determine the students’ 
needs, your team collaboratively identifies how the interventions can be implemented (Collaborative Climate) and by whom. 
For example, fractions can be incorporated not only in math class, but also in art or physical education classes. This 
creates multiple opportunities for students to interact with fraction concepts in several classes (Collaborative Use). 


As your intervention team discusses students, you consciously avoid using unnecessarily negative words like “slow” 

ot “incapable” to describe students. Instead, you focus on what the data are telling you and use constructive criticism 
and feedback both in verbal discussions about each student’s work and in each student’s documentation (E/hics). 

For example, you might use phrases such as “challenged,” “needs more practice in,” or “may not understand yet.” 
Throughout the process, your team works to understand and review, as necessary, any relevant legal requirements, such 
as the privacy requirements associated with special education information used in determining interventions (Rus 

and Laws). \n addition, your team members work to understand the appropriate policies to secure data. For example, 
leaving a flip chart in a public space with student names and performance is inappropriate (Rw/es and Laws, Protections). If 
rules are violated, you provide a safe environment for staff members to identify the issues and bring them forward for 
resolution (Use, Culture). 


When student interventions are determined, you reach out to parents via email or mail. The communications include 
a description of the assessment data that was used to determine the intervention (Transparency), as well as the specific 
interventions that will be used. It also includes information about how the parent can play a role in improving their 
student’s learning (Collaborative Culture). 


As is frequently the case with school initiatives, demonstrating professional behaviors should be an interactive and iterative 
process where teams are constantly revisiting each student’s performance, the effectiveness of current interventions, and 
the need to adjust instruction as necessary (Improving Outcomes). Data use professional behaviors are ingrained in all aspects 
of a successful school culture. 
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Key Terms 


As they are related to data or data use 


KEY TERMS are designated throughout the document. 


ACCOMMODATION 


ANALYSIS 


ASSESSMENT 


BASELINE 


BASELINE MEASURES 


BIAS 


CAUSAL 


CONFIDENTIALITY 


CONGRUENT 


CORRELATION 


CRITICAL EVALUATION/REFLECTION 
DATA ACQUISITION 


DATA COLLECTION 


DATA DEFINITION 


DATA DOCUMENTATION 


DESCRIPTIVE 
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An alteration of environment, format, or equipment that allows an 
individual to gain access to content and/or complete assigned tasks. 


The process of systematically applying statistical and/or logical techniques 
to describe and illustrate, condense and recap, and evaluate data. 


The systematic collection, review, and use of information about education 
programs undertaken for the purpose of evaluation or improvement. 


Assessment can also refer to an instrument, tool, or method of obtaining 
information from tests or other sources about the achievement or abilities 
of individuals (adapted from the National Council on Measurement in 
Education’s Glossary of Important Assessment and Measurement Terms, 
https://www.ncme.org/ncme/NCME/Resource_Center/Glossary/NCME/ 
Resource_Center/Glossary I.aspx). 


The level of performance at the start of data collection that can be used 
to measure change in indicators in the future. 

Variables or metrics used to identify an individual or group’s initial level 
of performance at the start of the data collection process prior to a 
program or intervention. 

Anything that produces systematic but unexpected variation resulting in 
inaccurate results. 

A type of data analysis used to try to determine a cause-and-effect 
relationship. 

The responsibility of a person who has access to another individual's 
personal data not to share the data without consent. 


Matching or in agreement with something. 


Describes when two variables are related, but where the relationship 

may not be causal. A positive correlation occurs when the values of two 
variables increase together (e€.g., years of education and income). A 
negative correlation occurs when one variable increases while the other 
decreases (e.g., number of absences and grades). 

Disciplined thinking that is informed by evidence. 

The process of collecting or gaining access to and organizing information. 
The process of gathering and measuring information in a systematic 
fashion to answer questions. 

Language for describing data or information structures. 

Information on the context of data collection, including collection methods 
(e.g., sampling, instruments, technology used); data sources; data validation 
and modification; and data confidentiality, access, and use conditions. 


A type of data analysis used to describe a set of data using mean, mode, 
median, distribution, and more. 
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FERPA 


FINDINGS 


FORMATIVE 


FORMATIVE ASSESSMENTS 


GENERALIZABILITY 


GROWTH MEASURES 


HARM 


HIGH-QUALITY DATA 


HIPAA 


HUMAN SUBJECTS 


INFERENTIAL 


MEASURE 


MULTIPLE MEASURES 


PREDICTIVE 


PRIVACY 


PSYCHOLOGICALLY SAFE 
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The Family Educational Rights and Privacy Act (FERPA) (20 U.S.C. § 12329; 
34 CFR Part 99) is a federal law that protects the privacy of student 
education records. The law applies to all schools that receive funds under 
an applicable program of the U.S. Department of Education. FERPA gives 
parents certain rights with respect to their children's education records. 
These rights transfer to the student when he or she reaches the age of 18 
or attends a school beyond the high-school level. Students to whom the 
rights have transferred are "eligible students." Learn more at http://www2. 
ed.gov/policy/gen/guid/foco/ferpa/index.html. 


The principal outcomes of a research project examining data, and what 
the project data suggested, revealed, or indicated. 


Describes information that can be collected and used to improve or 
inform progress during a learning opportunity. 


Tools or procedures used to collect information used to improve 
or inform progress during a learning opportunity. Also known as 
“assessments for learning.” 


The extent to which inferences from specific facts, statistics, and/or real- 
world examples can be applied to broader groups or populations. 


Measures comparing the relative change in performance of one person 
on a specific test with the change in performance of all others on the 
same test. 


Intentional or unintentional physical or mental damage or injury to 
subjects as a result of using subjects’ data. 


Data that are generally agreed upon as trustworthy and are gathered 
through a valid and reliable instrument or process. 


The Health Insurance Portability and Accountability Act of 1996. The 
HIPAA Privacy Rule standards implemented by the U.S. Department 

of Health and Human Services address the use and disclosure of 
individuals’ health information—called “protected health information” 
by organizations subject to the Privacy Rule (“covered entities”)—as well 
as standards for individuals’ privacy rights to understand and control how 
their health information is used. Learn more at https://www.hhs.gov/ 
hipaa/for-professionals/privacy/. 


U.S. Department of Health and Human Services regulations under 45 CFR 
46.102(f) define a human subject as a living individual about whom an 
investigator conducting research obtains (1) data through intervention or 
interaction with the individual: or (2) identifiable private information. 


A type of data analysis where a sample of data is used to say something 
about a larger population. This analysis usually requires some type of 
statistical model (€.g., two-sample t-test, z-test, two-sample z-test for 
proportions, ANOVA, chi square). 

A specific calculation, score, or classification that accurately represents 
the variable or characteristic being analyzed. 


A variety of measures, such as standardized test results, classroom 
assessments, tasks and projects, grades, and teacher evaluation, used to 
provide a complete picture of a student's academic achievement. 


A type of data analysis that uses current and historical facts to make 
predictions about the future. 

An individual's right to have his or her personal information protected 
from being accessed by others without permission. This right is balanced 
by the need for collection and dissemination. 


A shared belief that the group or context is safe for interpersonal risk 
taking involved in the collection, analysis, and sharing of data. 
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QUALITATIVE 


QUANTITATIVE 


RELIABILITY 
RELIABLE 


SCORE DISTRIBUTION 


SUMMATIVE 


SUMMATIVE ASSESSMENTS 


TIMELY 


TREND 


TRIANGULATION 


VALID 
VALIDITY 


VARIABLE 


al 


Data that are not expressed numerically (e.g., interviews, student 
projects, student behavior, videos). 


Data that are expressed numerically (e.g., assessment scores, GPA, 
attendance, enrollment, surveys). 


The degree to which an assessment tool produces stable and consistent 
results given similar conditions. 


Stable and consistent results. 


The dispersion of test scores plotted by frequency along an ordinal or 
interval-ratio scale. 


Describes assessments used to evaluate learning, skill acquisition, or 
achievement at the conclusion of a defined instructional period. 


Tools or procedures used to collect information to evaluate learning, skill 
acquisition, or achievement at the conclusion of a defined instructional 
period. Also known as “assessments of learning.” 


Data that are collected, analyzed, and reported at the most useful time 
allowing future decisions or actions to be made. 


The general direction in which something is developing or changing. 


Using more than one approach to answer a research question, or using 
more than one instrument or way of measuring something to increase 
confidence in the findings. 


Measures what it is supposed to measure. 


The degree to which an assessment tool measures what it is supposed 
to measure. 


Any characteristic, number, or quantity that can be measured or counted. 
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Appendix A: Data Use Standards 
Workgroup Members 


Po State | Nome | gcnization 


California Christina Tydeman Dominican University of California 

Guam Zeni Natividad Guam Department of Education 

Hawaii Justin Katahira? Hawaii P-20 Partnerships for Education 

Hawaii Christine Sorensen Irvine Pacific Regional Comprehensive Center; University 
of Hawaii at Manoa 

Idaho Andy Mehl Idaho Board of Higher Education 

Illinois Doug Franklin Illinois Board of Higher Education 

Kansas Eric Punswick Olathe Public Schools 

Kansas Lori Adams Kansas State Department of Education 

Montana Brett Carter Montana Office of Public Instruction 

Nebraska Russ Masco? Nebraska Department of Education 

Nebraska Dick Meyers University of Nebraska at Kearney, 
College of Education 

New Hampshire Irene Koffink New Hampshire Department of Education 

New Hampshire Mike Schwartz New Hampshire Department of Education 

New York David Weinberger Yonkers Public Schools 

North Dakota Nathan Anderson Mid-Dakota Education Cooperative 

North Dakota Magdalena Brockel Missouri River Educational Cooperative 

North Dakota Stacy Duffield North Dakota State University 

North Dakota Amy Engelhard Education Technology Council 

North Dakota Jen Glasheen Southeast Education Cooperative 

North Dakota Steve Snow North Dakota Department of Instruction 

Oklahoma Bryan Duke University of Central Oklahoma 

Oklahoma James Machell University of Central Oklahoma 

Oklahoma Susan Pinson Oklahoma State Department of Education 

South Dakota Sara Kock?‘ South Dakota Department of Education 

Tennessee Margie Johnson® Metro Nashville Public Schools 

Washington Melissa Beard Education Research and Data Center 

Washington Jim DePaepe Central Washington University 

Facilitator Corey Chatis SLDS Grant Program State Support Team 


? Professional Behaviors Subgroup Lead 
3 Skills Subgroup Co-Lead 

* Knowledge Subgroup Lead 

° Case Studies Subgroup Lead 


Knowledge, Skills, and Professional Behaviors tor Effective Data Use - Version 2, July 2010 


Appendix B: Bibliography of 
Resources Consulted® 


Bernhardt, V.L. (2003). Using Data to Improve Student Learning in Elementary Schools. Larchmont, NY: Eye on Education. 


Council for the Accreditation of Educator Preparation. (2013). CAEP 2013 Standards for Accreditation of Educator Preparation. 
Available at hitp:// caepnet.org/standards/ introduction 


Danielson, C. (2013). The Framework for Teaching. 
Available at bitp://danielsongroup.org/userfiles/ files /downloads/ 2013 EvaluationInstrument.pdf 


Danielson, C. (2014). Correlation between the Danielson Framework for Teaching and the Interstate Teacher Assessment and Support 
Consortium (InTASC) Standards. Available at hitp:/ / danielsongroup.org/ download] ¢download=452 


Data Quality Campaign. (2012). Data Quality Campaign 2012 Data for Action Survey. 
Available at bitp:// www.dataqualitycampaign.org/ files] DEA2012_Survey_Final.pdf 


Gummer, E.S., & Mandinach, E.B. (2015). Building a Conceptual Framework for Data Literacy. Teachers College Record, 117(040305). 


Interstate Teacher Assessment and Support Consortium (InTASC). (2011). InTASC Model Core Teaching Standards: A Resource for 
State Dialogue. Available at http://wwm.cesso.org/Documents/2011/InTASC_Model_Core_Teaching_Standards_2011 pdf 


Kekahio, W., & Baker, M. (2013). Five Steps for Structuring Data-Informed Conversations and Action in Education (REL 2013-001). 
US. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional 
Assistance, Regional Educational Laboratory Pacific. Available at hitp://ies.ed,gov/ pubsearch/ pubsinfo.asp ¢pubid=REL201 3001 


Learning Point Associates. (2004). Guide to Using Data in School Improvement Efforts: A Compilation of Knowledge from Data Retreats 
and Data Use from Learning Point Associates. Available at hitp://mww.learningpt.org/pdfs/ datause/guidebook.pdf 


Love, N. (2008) Building a High Performing Data Culture. The Data Coach’s Guide to Improving Learning for All Students: 
Unleashing the Power of Collaborative Inquiry. Available at bitp://wwm.sagepub.com/upm-data/27678_Love___Chapter_1.pdf 


National Forum on Education Statistics. (2010). Forum Guide to Data Ethics. (NFES 2010-801). U.S. Department of Education. 
Washington, DC: National Center for Education Statistics. Available at hitp://nces.ed.gov/forum/pub_2010801 asp 


National Forum on Education Statistics. (2004). Forum Guide to Protecting the Privacy of Student Information: State and Local Education 
Algencies (NCES 2004-330). U.S. Department of Education. Washington, DC: National Center for Education Statistics. 
Available at bitps://nces.ed,gov/ pubs2004 / 2004330.pdf 


National Forum on Education Statistics. (2012). Forum Guide to Taking Action with Education Data. (NFES 2013-801). 
US. Department of Education. Washington, DC: National Center for Education Statistics. 
Available at bitp://nces.ed.gov/forum/pub_2013801.asp 


National Council for Accreditation of Teacher Education. (2008). NCATE Unit Standards. 
Available at bitp://www.ncate.org/ Portals/0/documents/Standards/ NCATE%20S tandards%o202008 pdf 


Nebraska Department of Education. (2013). Data Literacies, Concepts, and Indicators. 
Available at bitp://wwm.education.ne.gov/ DataS ervices/ SLDS_Grant/ Data_Literacies.pdf 


° Additional resources were consulted in addition to those listed here. The listed resources had the greatest influence on the document. 


32 SLDS Data Use Standards 


Oregon Direct Access to Achievement (DATA) Project. (2012). Certification Resources. 
Available at bitp://wwm.oregondataproject.org/ content/ certification-resources 


Oregon Direct Access to Achievement (DATA) Project. (2014). Data Retreat Resources. 
Available at htp://www.oregondataproject.org/content/ data-retreat-resources 


Teacher Leadership Exploratory Consortium. (2012). Teacher Leader Model Standards, Domain V: Promoting the Use of Assessments 
and Data for School and District Improvement. Available at http:/ /wwu.teacherleaderstandards.org/the_standards_domain_5 


US. Department of Education, Office of Planning, Evaluation and Policy Development. (2011). Teachers’ Ability to Use Data to 
Inform Instruction: Challenges and Supports. Available at hitp://www2.ed.gov/ rschstat/ eval/ data-to-inform-instruction/ report.doc 


US. Department of Education, Office of Planning, Evaluation and Policy Development. (2010). Use of Education Data at the 
Local Level From Accountability to Instructional Improvement. 
Available at bitp://www2.ed.gov/ rschstat/ eval/ tech/ use-of-education-data/ use-of-education-data.doc 


Wayman, J. (2013). The Data-Informed District: Research on How Educators Use Data to Inform Practice. US. Department of 
Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional 
Educational Laboratory Southwest. Available at Aip://relsouthwest.sedl.org/ bridge_events/2013-12-05_data_use/ index.html 


Wayman, J., Jimerson, J.B., and Cho, V. (2012). Organizational Considerations in Establishing the Data-Informed District. 
School Effectiveness and School Improvement: An International Journal of Research, Policy and Practice. Volume 23, Issue 2. 
Available at bitp://do.doi.org/10.1080/09243453.2011.652124 


Wayman, J., et al. (2012). Using Data to Inform Practice: Effective Principal Leadership Strategies. Presented at the annual 2012 
American Educational Research Association in Vancouver, British Columbia, Canada. 

Available at bitp:// edadmin.edb.utexas.edu/ datause/papers/ Wayman%20Spring/20Lemke%20Lehr%o20Principal¥o20Data%20Use%20 
Strategies.paf 


Knowledge, Skills, and Professional Behaviors for Effective Data Use - Version 2, July 2010 80 


